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In a haptic search task, one has to detect the presence of a target among distractors using the sense of touch. 
A salient target can be detected faster than a non-salient target. However, little is known about the 
exploration strategies that are used, especially in 3D search tasks where items are held in the hand. In this 
study, we investigated which parts of the hand were used to contact the target and which strategies were 
performed. Blindfolded participants performed search tasks in four conditions, where the targets differed in 
relevant property and saliency. The positions of the target and the hand were tracked during exploration. It 
was found that target saliency had a large effect on the use of the hand parts and the strategies. In the 
non-salient target conditions, the fingers, especially the thumb, contacted the target more often than in the 
salient target conditions. This could also be seen in the strategies, where the thumb was used to explore 
the items in a serial way by moving them in the hand or touching them individually. In the salient target 
conditions, more parallel strategies like grasping or shuffling of the items in the hand were used. 

Humans are very skilled in exploring multiple objects at the same time by touch. With a handful of objects, a 
desired object can usually be found quite well among the others. The search for your keys in your bag, or 
for a specific button on a control panel, are both examples of haptic search tasks. In a search task, one 
searches for a desired object (the target) among other objects that are not of interest (the distractors). Some 
features can be detected immediately among other items that do not possess that feature. This is called the pop-out 
effect 1 . The feature that is present in the target is then said to be salient with respect to the distractor property and 
to be processed very efficiently. In vision research, search tasks have been widely used to investigate the processing 
of different features. Recently, there has also been a growing interest in the search capabilities and efficiency using 
the haptic modality 2 7 . 

When searching for a target, many different search strategies can be used. Usually, a division is made between 
parallel strategies (explore items all at once) and serial strategies (explore items one at a time). This is often based 
on the reaction time, which increases with the number of items in a serial search, giving a positive search slope. 
With a parallel strategy, the search will be independent of the number of items and thus give a flat search slope. In 
visual search, a variation in eye movements can also be seen 8 , but in haptic search, with the many degrees of 
freedom in the hand, the number of possible strategies might be much larger. 

In 2D haptic search tasks, where the items are placed on a display, some investigation of the performed 
explorations has been done. Differences in velocity profiles 9 , normal force 10 , speed and travelled distances 3,6 were 
found between tasks that differed in search difficulty. When trials were classified into movement types according 
to quantitative measures, it was shown that more simple strategies were used with a salient target and more 
detailed strategies in a more difficult search 11 . 

In contrast to these 2D hand analysis, the detailed analysis of movement strategies in 3D haptic search tasks has 
yet to be performed. In previous studies, where handheld items were searched, only the percentage of trials in 
which items that are dropped out of the hand were calculated 4,712 , but this gave only limited information about the 
strategies that were used. Until now, there has been little attempt to quantify these strategies in the search for 
handheld objects. 

In fact, the manipulation of multiple items in the hand is a topic that has been rarely studied. In the context of 
haptic exploration of single objects, movements have been analysed using video analysis 13 or quantitative mea- 
sures 14 . Haptic perception is an active process, so also when multiple objects are present in the hand they need to 
be moved through the hand to perceive most of their properties. With multiple items, the exploration movements 
might also depend on the perceived property, but other issues like keeping track of which item is felt and the 
handling of items that occlude each other must be handled as well. 

Another interesting aspect of haptic search is that the hand that is used to perform this task contains many 
different sensors that are spread unevenly over parts of the hand. For instance, the fingertips are more densely 
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Table 1 The reaction times and percentage of errors in each con - 
dition, shown for target-present and target-absent trials separately. 
Reaction times are means ± standard errors of the mean. Error 
percentages are calculated as the number of errors divided by 
the number of trials + the number of errors* 




Reaction times (s) 


Errors {%) 




present 


absent present absent 


cube 
sphere 
rough 
smooth 


1.12 ± 0.13 
2.28 ± 0.15 
1.26 ± 0.09 
3.68 ± 0.38 


2.36 ± 0.30 
6.11 ± 0.55 
2.54 ±0.19 
8.54 ±0.86 


1 0 
8 0 
3 0 
12 0 


*Percentages 


were rounded to integers. 







populated than the palm of the hand 15 , making them more sensitive 
to small differences in properties. This does not mean that in a haptic 
search task the hand palm would not contribute to the perception of a 
target. An interesting question, then, is which parts of the hand 
contribute to the search. One can imagine that for more difficult 
searches, the fingers are used more extensively to explore the items 
than in easy searches. 

In the present study, we had two aims in analysing the exploratory 
movements that are used in a haptic search task. First, we were 
interested in which parts of the hand were used to perceive the target. 
The second objective was to analyse which strategies are used to 
perform the task. We investigated four search conditions: two in 
which the target differed from the distractors in shape (spheres 
and cubes) and two in which it differed in texture (rough and 
smooth). In either group there was a difficult search (sphere among 
cubes or smooth among rough) and an easy search (cube among 
spheres or rough among smooth). The easy search tasks were for 
salient targets and the difficult search tasks were for non-salient 
targets, as known from previous research 3,412 . 

In each of the four conditions, participants had to quickly and 
accurately determine whether a target was present or not. The time 
it took until they gave a response was the reaction time. Furthermore, 
the hand movements were measured by taping sensors to the hand. 
From these data, a model of the hand was made 16 . With this model, a 
contact analysis was performed to determine which parts of the hand 
contacted the target at what time. 

The second analysis was the investigation of the movement strat- 
egies that were used in the search. Four strategies were defined and 
each trial was classified by a set of variables into one of those four. 
The strategies were named shuffle, grasp, manipulate and thumb and 
could be described as follows: 

• Shuffle was a movement in which the items were shuffled in the 
hand. They were not felt individually, but quickly moved within 
the hand. 

• Grasp was a grasp of the stimuli, in which no fine movements 
were made. This was usually a short movement, in which the 
answer was given immediately. 

• Manipulate was a movement in which the items were moved 
individually with the thumb in the hand, more or less one by 
one, and sometimes dropped out of the hand. 

• Thumb was a strategy where the items laid in the outstretched 
hand and the thumb individually touched the items, one by one. 

These strategies were based on the observation of the trials and 
that of previous studies. They were chosen because they were regu- 
larly observed and were also distinct enough to allow the trials to be 
grouped into different categories. Therefore, the four strategies 
should be seen as different classes, where a variation of movements 
is possible but still belonging to the same group. 

Results 

Since the analysis of the contact with the target yielded many signifi- 
cant results, this section will only focus on the most prominent 
results. The full report of the results can be found in the Supplemen- 
tary Information. 

Reaction times and errors. The reaction times and errors are 
presented in Table 1. As can be seen, lower reaction times and 
error percentages were found in the search for a cube and a rough 
target than in the smooth and sphere conditions. 

Contact with the target. The segments of the hand that were 
analysed were the hand palm, and three phalanges (distal, middle 
and proximal) of each finger, including the thumb. The percentage of 
trials in which a segment was in contact with the target at any 
moment in the trial is displayed in Fig. 1. In 5 trials (0.5%) no 



contact was found with any hand part during the trial. This was 
only in the conditions with a rough target or a cubical target. 

It was not desirable to conduct one Analysis of Variance 
(ANOVA) on all the segments, since the hand palm segment did 
not consist of different phalanges whereas the different fingers do. 
Treating each segment as a separate factor would have made the 
interpretation quite difficult as well. Therefore, we decided to com- 
pare the four segment groups (distal, middle and proximal phalanx 
and hand palm) with each other and subsequently analyse each 
segment group separately. The ANOVA values can be found in the 
Supplementary Information, in Tables SI -3. 

As can be seen in Fig. 1, it is clear that the percentage of target 
contacts differs between the different parts of the hand. A 4 (segment 
group) X 4 (condition) repeated measures ANOVA indicated that 
the hand palm and distal segment had the most contact with the 
target. 

To evaluate the differences between the fingers, the distal, middle 
and proximal segments were evaluated separately. The target contact 
with the distal segment was further examined in a 5 (finger) X 4 
(condition) repeated measures ANOVA. Effects of finger, condition, 
and an interaction effect of finger X condition were found. The index 
finger and the little finger seemed to make less contact with the target 
compared to the other fingers. Differences between the fingers were 
more prominent in the sphere and smooth conditions. The main 
effect of condition in the distal phalanx indicated that the target 
was touched less often in the cube and rough conditions than in 
the sphere and smooth conditions. The interaction effect of con- 
dition X finger indicated that the conditions differed primarily in 
the use of the thumb, and also in the index and little finger. In Fig. 1, 
these differences between the conditions in the amount of target 
contact with the thumb can be clearly seen. 

The analysis of contact with the middle phalanx from a 5 (finger) 
X 4 (condition) repeated measures ANOVA revealed effects of fin- 
ger, condition, and an interaction effect of finger X condition. The 
middle and ring finger were contacted more than other fingers, but 
differences were only significant in the sphere and smooth condi- 
tions. Similar condition differences as for the distal phalanx were 
found, but primarily in the little finger and also in the middle finger. 

For the proximal phalanx, a 5 (finger) X 4 (condition) repeated 
measures ANOVA was performed. An effect of finger was found. 
Post-hoc tests revealed that the thumb less frequently touched a 
target than the index, middle and ring finger. In addition, less contact 
was observed for the little finger compared to the middle and ring 
finger. There was no effect of condition or an interaction effect. A 
repeated measures ANOVA on the hand palm contact data showed 
no significant effect of condition. 

In sum, the hand palm and the distal segment most often made 
contact with a target. Differences between the conditions seemed to 
exist mainly in the distal and middle segments, with more contact in 
the sphere and smooth conditions than the rough and cube condi- 
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Figure 1 | The percentage of trials in which a segment was in contact with the target in the trial. Each panel (condition) shows data for the distal (dist), 
middle (midd) and proximal (prox) segments, for each finger (thumb, index finger, middle finger, ring finger and little finger), and for the hand palm 
segment. Error bars indicate standard errors. The top right of each panel shows the data plotted on a hand model. Darker colours indicate a higher 
percentage of trials with contact. 



tions. For the distal segment, the conditions differed primarily in the 
use of the thumb, whereas they differed most in the little finger for the 
middle segment. Most differences between the fingers were found in 
the smooth and sphere conditions. 

The amount of contact might also differ over the course of the trial. 
To investigate this, the target contact was analysed at different 
moments in time. Each trial was divided into 20 intervals, to examine 
changes over the course of the trial. When the contact points with the 
targets were inspected at different time intervals, no clear patterns 
were seen in the middle and proximal segments or the hand palm 
segment. In these segments, the amount of contact did not differ 
much over time. In the distal segment, differences in time were 
found, as can be seen in Fig. 2. At the end of the trial, the percentage 
of trials in which a target was contacted seemed to increase rapidly, 
primarily for the thumb in the smooth and sphere conditions. 

A 5 (finger) X 4 (condition) X 2 (time step) repeated measures 
ANOVA was performed on the data of the distal segment. The first 
and last interval were chosen as time steps to examine the differences 
at the start and end of the trial. All main effects and interactions were 
significant. The finger and condition effect replicated results 
described above and can be found in the Supplementary 
Information. The effect of time step indicated that more finger con- 
tact was seen at the end of the trial than at the beginning. This 
increase in amount of contact with time was significant for the index 
finger in the rough, sphere and smooth conditions. In the cube con- 
dition, an increase was found for the middle finger and thumb. As 
can be clearly seen in Fig. 2, this increase in contact was also signifi- 
cant for contact with the thumb in the sphere and smooth condition. 



In the analysis, even a contact point at a single sample counted as a 
touch of the target. One sample corresponds to 10 ms, so this might 
not reflect an actual manipulation of the target. Therefore, an ana- 
lysis was performed where the contact points were checked in a 
window of 100 ms. If in at least 70% of the samples in a window 
the target was contacted, this was seen as a long contact period. If 
only these longer contact periods were used in the analysis, similar 
results were found as described above. Overall, the percentage values 
were somewhat lower, but the patterns remained the same. 

Movement classification. The second aim of the study was to 
examine the exploration strategies. Trials were classified into the 
four different strategies as described in the Introduction. Examples 
of the strategies are illustrated in Fig. 3 to give some indication of the 
exploratory motions. These are movie stills from videos that can be 
found in the Supplementary Information. A set of criteria were 
chosen to distinguish between the strategies. The criteria variables 
were the speed of the thumb and the ring finger, the volume in which 
the thumb moved and the bending and stretching of the ring finger. 
The expected patterns are indicated in Table 2. The results are shown 
in Fig. 4. The grasp strategy was used most often, especially when the 
target was present. For the cube and rough conditions, this strategy 
was used primarily, whereas in the sphere and smooth conditions 
different kinds of strategies were also frequently seen. When the 
target was absent, larger differences between the conditions were 
found. In the sphere and smooth conditions, most often the 
manipulate strategy was used, whereas most trials in the rough and 
cube conditions showed grasp strategies. 
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Discussion 

In this study we investigated the explorations that are used in a haptic 
search task. Specifically, we focused on two aspects. The first aim was 
to study which parts of the hands are used. Secondly, the strategies 
that are used in the explorations were of interest. We investigated 
these aspects in four search tasks. Two properties, shape and texture, 
were used to distinguish the target from the distractors. 

Previous research has shown that edges and vertices (present in 
cubes) and roughness are salient features 2-4,12 . Similar to those stud- 
ies, we found shorter reaction times and a smaller number of errors in 
the search for cube among spheres or a rough sphere among smooth 
spheres than in the search for the reversed situations. These results 
confirmed, as a baseline, that the searches for cubes and rough 
spheres were easier than the searches for spheres and smooth 
spheres. This was used to further investigate the exploration move- 
ments that were used in these search tasks. 

For the first objective of this study, we examined which parts of the 
hand were used in the search for a target. Therefore, we checked 
which parts of the hand were in contact with the target during the 
exploration. There were some general observations with respect to 
target contact that did not depend on the kind of target. Results 
indicated that especially the hand palm and the fingertips were used 
to perceive the target. The middle and proximal phalanges made less 
contact with the target. This is probably because all the items are held 
in the hand with bent fingers. The middle and proximal phalanges 
are folded together and thus do not contact the target often. 

The differences in the amount of target contact between the search 
for a shape or a texture feature did not appear to be very large. The 
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Figure 3 | Movie stills of four examples of the movement strategies. From 
top left: shuffle, grasp, manipulate, thumb. The movies with the 
corresponding hand model can be found in the Supplementary 
Information. 
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speed thumb 


high 
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high 


high 


speed ring finger 


high 


low 


high 


low 


volume thumb 


low 


low 


high 


high 


turning points 


high 


low 


high 


low 



saliency of the target seemed to account for most of the differences 
between the search conditions: if the target was not salient, as in the 
sphere and smooth conditions, more target contact was seen than in 
the conditions with a salient target (rough and cube). The differences 
between the conditions were only found in the use of the distal and 
middle parts of the fingers. The difference in the percentage of thumb 
contact between the conditions was especially large. The higher 
amount of contact indicates that the fingers were used more often 
to perceive the target in these conditions. This is a reasonable result, 
because the fingers are more densely innervated with tactile receptors 
than other parts of the hand 15 and are thus more sensitive. Indeed, it 
might be that only once the fingers touch the target it is identified, 
whereas for an easily perceivable target, contact with less sensitive 
parts of the hand might already be enough. 

This important role of the fingertips was also seen when looking at 
the moment of contact. At the end of the trial, the thumb was more 
often contacted than at the beginning in the conditions with a non- 
salient target. Such behaviour suggests that once the thumb touches 
the target, it is identified and a response can be given. In the current 
task, with handheld items, the thumb can easily move over and touch 
an item to identify it. 

A few issues must be noted when interpreting the contact with the 
target. With the longer trials, as in the difficult tasks, the chance to 
accidentally touch the target with a certain hand part is higher. 
However, similar results were found with longer contact durations. 
In addition, clear strategy differences were seen between the tasks, 
which indicate a change in how the fingers are used. Therefore, we 
believe the contact differences are not due to chance. Secondly, it 
must be noted that the fact that the participant touched the target 
does not mean that this contact is consciously perceived, for instance, 
due to light touches or attentional focus. This makes it difficult to 
relate the contact to the participants' response. However, if a target is 
touched more often with a specific hand part, it is likely that it is also 
consciously perceived more often with that hand part. Therefore, we 
think that our results do reflect how the separate hand parts are used 
and that these parts are probably even necessary to perform the task. 

The second analysis investigated the strategies that were used in 
the search tasks. Similar to the use of the various hand parts, no clear 
differences were seen between the search for a texture or shape fea- 
ture in the exploration strategies. This might seem surprising, 

target present 
shuffle grasp manipulate thumb 



because very different exploratory procedures are optimal for these 
properties, i.e., enclosure for shape and lateral motion for textures 13 , 
although these exploratory procedures were defined for larger 
objects. Possibly, the roughness of the rough target was so obvious 
it could be felt with just a small movement, not clearly visible as a 
different strategy. In addition, the choice of the strategy categories 
might not be sensitive enough for the differentiation between 
searches for the two property dimensions. 

Nevertheless, the movement classification showed clear strategy 
differences between the different tasks with respect to the target 
saliency. In searching for a cube or a rough sphere among smooth 
spheres, a single grasp was often enough to detect the presence of the 
target and the grasp strategy was used most. In searching for a non- 
salient target, this strategy was less often used, and especially when 
the target was absent, more extensive and serial explorations were 
necessary, as seen in the frequent use of the manipulate strategy in 
these conditions. This difference of simple explorations in easy 
searches compared to more detailed explorations in difficult searches 
was also seen in previous search studies that used 2D set-ups 3,611 . 
Participants did not know whether a target was present or not and 
probably did not adjust their strategy much to the presence of the 
target, but they could stop their exploration once the target was felt. If 
a detailed exploration was intended, but the target was felt early in the 
search, the exploration could be aborted in a target-present trial. In a 
target-absent trial, the movements would have continued. Although 
the trials would be classified differently, the intention might have 
been the same. 

These results confirm the assumption that parallel strategies can 
be used if the target is salient, but not when it is difficult to detect. 
This is the first time this is shown in a quantitative movement ana- 
lysis in 3D search tasks. Previous research mainly used search slopes 
to distinguish between parallel and serial strategies. Our results indi- 
cate that these commonly found differences in search slopes are 
indeed caused by the performed strategies. Interestingly, these clear 
differences between search tasks can already be made apparent by 
studying a single number of items. In the tasks with a salient target, a 
rough or cubical target, a grasp or a shuffling movement was almost 
always used. Both of these exploration patterns can be considered 
parallel strategies. In the search for a smooth or spherical target, 
detailed serial strategies are performed, in which the thumb is used 
to individually search the items. 

The movement classes were chosen based on previous research 
and observation. Of course, many more strategies could be used and 
intermediate behaviours were also seen. This also emphasises that 
parallel and serial strategies can exist in many forms. The four strat- 
egies reflect some of the possible exploration categories that are used 
in search tasks and the criterion variables were chosen to distinguish 
between these categories. Overall, the differences between the con- 
ditions are large enough to give an insight into the influence of target 
saliency on the exploration strategies in haptic search. Future 

target absent 
shuffle grasp manipulate thumb 



cube 24 73 0 3 cube 27 70 2 2 



sphere 16 59 16 10 sphere 18 6 70 7 



rough 15 82 2 1 rou g h 21 76 2 1 



smooth 8 51 25 16 smooth 7 2 84 6 



Figure 4 | Percentage of trials of the classified strategies in each condition, for target-present and target-absent trials separately. Numbers are sized 
proportionally to their values, so greater sizes are used for higher percentages. 
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research could aim to improve the classification. A useful addition 
would be to look at the strategy usage over time, so switches could be 
made apparent. The current criteria are not suitable for such an 
analysis, but serve as a first step in the classification of search beha- 
viour. Another topic for future studies might be to look at the strat- 
egies used for the localisation of objects, which is also important in 
daily life searches. 

The kind of strategy that is used influences which parts of the hand 
contact the target. For example, in a grasp strategy, the fingertips will 
not touch the target in most cases. This strategy was most often used 
in the conditions with a salient target, a cube or a rough sphere, and 
also the amount of target contact was low in these conditions. On the 
other hand, which parts of the hands need to be used can also deter- 
mine the strategy choice. It seems that the more sensitive parts of the 
hand, i.e., the fingertips, need to be used to detect the presence of a 
non-salient target. To use the fingertips to touch the target, the items 
need to be shuffled through the hand. In the case of a difficult target, 
the items might need to be searched in a serial manner. To keep track 
of which item is felt, they must be explored in a coordinated way. For 
instance, they might be dropped out of the hand one by one, or felt 
individually in the hand. These strategies were seen more often in the 
conditions with non-salient targets, the smooth spheres among cubes 
or rough spheres. In these conditions, the thumb was used often to 
contact the target. The thumb has a wide range of motion and can be 
easily used to move items through the hand or explore items that are 
arranged on the hand palm. Our study investigated in-hand manip- 
ulation. In other tasks where, for instance, the hand explores objects 
more from above instead of grasping, slightly different strategies and 
contacts might be seen. However, we expect the difference between a 
parallel or serial strategy and increased contact for difficult tasks 
compared to easy tasks to remain. 

In conclusion, humans seem to be very skilled to adjust their 
movements to the specific task, also in the exploration of multiple 
objects. In a haptic 3D search task the use of hand parts for explora- 
tion of the stimuli depends on the saliency of the target. This is the 
first study that has investigated the exploration behaviour of a 3D 
haptic search task in detail and in a quantitative manner. This 
information can be useful in the development of tactile sensors for 
robots or haptic devices. Although many different sensors exist, still 
much can be learned from human data to improve robotic 
designs 17 ' 18 .The placement and sensitivity of the sensors as well as 
the movement strategies could be adjusted to the specific task that is 
performed with such a device. 

Methods 

Additional information about the apparatus, methods and analysis is described in the 
Supplementary Information. 

Participants. Ten participants (6 females, 22 ± 3 years old) took part in the 
experiment. All participants were right-handed according to Coren's test 19 and used 
their right hand for performing the experiment. Participants signed an informed 
consent form prior to the experiment. This study was conducted in accordance with 
principles as stated in the declaration of Helsinki and was approved by the Ethics 
Committee of the Faculty of Human Movement Sciences of the VU University, 
Amsterdam. 

Apparatus. The hand movements were measured with a 3D Guidance TrakSTAR 
movement tracking system {Ascension Technology Corporation), that measured 6 
degrees of freedom. One sensor was placed on the nail of each of the five fingers, 
including the thumb, and one sensor was placed on the back of the hand, 
approximately in the middle in line with the knuckle of the middle finger. Two 
sensors were placed inside the stimuli (see below). 

Three types of wooden stimuli were used in the experiment: rough spheres, smooth 
spheres and smooth cubes (Fig. 5). The rough spheres were created by gluing small 
pieces of sandpaper (Bosch, P60) on the spheres 12 . A stimulus set for a single trial 
always consisted of seven items. This number was chosen because this was the largest 
number that could still fit comfortably in the hand and greater strategy differences 
between the conditions can be expected with a larger number of items. 

Two stimuli were attached to the sensors and the other five stimuli were attached to 
dummy wires, made from rubber tubes. The sensor wires were coated with the same 
material as the dummy wires, so all wires felt the same. In trials with no target present, 




Figure 5 | The experimental set-up. Top: the stimuli hang from wires 
above a resting cushion. Below: the stimuli; a smooth cube, a smooth 
sphere and a rough sphere. 

two distractors contained a sensor. When the target was present, one sensor was 
placed in the target and one in a distractor. 

Procedure. There were four conditions: a smooth cube among smooth spheres 
(cube), a smooth sphere among smooth cubes (sphere), a rough sphere among 
smooth spheres (rough) and a smooth sphere among rough spheres (smooth). In each 
condition, the procedure was similar. Participants had to determine whether a target 
was present or not. The identity of the target and the distractors was explained before 
the start of a condition. When the measurement was started, they lifted their hand to 
initially grasp the bunch of stimuli, which started the reaction time measurement. 
After that, they were free to manipulate the stimuli in any way they preferred. 
However, they had to be as fast as possible, but also make as few mistakes as possible. 
They indicated their answer by calling out 'yes' or 'no'. Their response, as recorded 
with the microphone of a head-set that was placed on the participants' head, stopped 
the measurement. They received feedback about the correctness of their answer. 
Erroneous trials were repeated at the end of the condition. Participants were 
blindfolded during the experiment. Preceding a condition, participants performed a 
practice block. 25 target -present trials and 25 target-absent trials were presented in a 
randomised order. The order of the conditions was randomised among participants. 

Hand model. To be able to evaluate which parts of the hands were used in the search 
and to investigate the movement strategies, a model of the hand was used 16 . The hand 
was modelled as a number of connected segments, where the hand palm consisted of 
one rigid body and the fingers and thumb had 3 segments each; one for each phalanx 
(distal, middle and proximal). The joint positions were calculated from the sensor 
positions and the segments were constructed around these joint parts. 

A detailed description of the calculations to construct the hand model can be found 
elsewhere 16 . The joint positions for the distal joint (near the fingertip) were calculated 
by transforming the nail sensor coordinates. The knuckle joint was determined by the 
transformation of the coordinates of the sensor on the back of the hand. The proximal 
joint (between middle and proximal phalanx) was determined by solving a set of 
equations. Here, a triangle was assumed between the distal joint, the proximal joint 
and the knuckle joint. 

The segments were constructed by transforming the sensor coordinates or by 
summations of vectors around the calculated joint coordinates. In this way, each 
segment was formed as a rigid block with 8 vertices. An example of the modelled hand 
is shown in Fig. 6. 

The constant measures of the hand (length, width and thickness of the segments) 
and the initial orientation of the sensors were determined from a photograph of the 
hand and a calibration procedure 16 . 
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Figure 6 | An example of the modelled hand including a target (black 
sphere). For the little finger, the distal, middle and proximal phalanx are 
specified. For the thumb, the distal, proximal and knuckle joint are 
indicated. The distal and proximal joints were located inside the fingers, 
but are drawn on top for illustration purposes. 

Analysis. The time between the start (touch of stimuli) and end of the trial (vocal 
response) was the reaction time. Only correctly answered trials were included in the 
analysis. Two analyses were performed for the investigation of the hand motions. In 
the first analysis, the contact with the target of different hand parts was evaluated. 
Here, only target-present trials were used, since only these trials contained a target. In 
the second analysis, the movement strategies that were used for exploration were 
investigated. This analysis included both target- present and target-absent trials. 

Analysis of target contact. In this analysis, we calculated which parts of the hand 
were in contact with the target during exploration. The model of the hand was used to 
determine the location of all parts of the hand. To see where the target touched the 
hand during the trial, the distance of the target to the 6 planes of each segment (five 
distal phalanges, five middle phalanges, five proximal phalanges and the palm of the 
hand) was calculated. If the distance to one or more of the planes of a segment was 
equal or less than the target radius, this was counted as a contact point with this 
segment. The target was always assumed to be spherical with a radius of 7.5 mm. A 
detailed description of the calculations can be found elsewhere 16 . In addition to these 
calculations, we also counted a contact when the distance to the line segments of the 
planes was smaller than the target radius. 

Subsequently, we determined whether a segment had made contact with the target 
during the trial. The percentage of trials in each condition where this happened was 
calculated for each segment. For the distal segments (fingertips) we also calculated at 
what moment in time the target was touched. The trial was divided into 20 intervals 
with a length of 5% of the reaction time. In each interval we determined whether a 
finger made contact with the target. 

Movement classification. To classify the movements, four different strategies were 
defined: shuffle, grasp, manipulate and thumb. The set of criteria variables that was 
used to classify the movements is shown in Table 2: the median speed of the thumb 
and ring finger with respect to the hand, the volume of the convex hull the thumb 
positions enveloped and the number of turning points {from the angular velocity) of 
the ring finger. 

All variables were normalised by dividing by the mean value over all conditions for 
each participant separately. The normalised variable set for each trial was matched to 
the pattern of high and low values for each strategy, as described in Table 2. The 
pattern of a strategy was represented as 0.7 for low values and 1.3 for high values (30% 
below and above the mean). For example, the pattern of the thumb strategy would be 
[1.3, 0.7, 1.3, 0.7]. For each strategy, the pattern was subtracted from the calculated 
values of the trial that was analysed. The sum of squares was calculated for these 
differences and the strategy with the smallest sum of squares was chosen for that trial. 
In this way, each trial was assigned one strategy. The automatic classification was not 
always similar to human observation (see also the Supplementary Information), but a 
human observer is also subjective and should not be seen as the standard. 

Statistics. The statistics are described in the Results section. In all analyses, an a-value 
of 0.05 was used. If Mauchley's test of sphericity was violated, a Greenhouse-Geisser 
correction was applied. Post-hoc tests were performed using paired f -tests. A 
Bonferroni correction was applied to relevant comparisons that were chosen in 
advance. Effect sizes are given as partial eta squared (ffp. 
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